[scala-sttp4] Omit optional fields set to None instead of serializing null (circe) - #24362
Conversation
… null (circe) The circe branch of scala-sttp4 uses plain deriveEncoder, which serializes Option None as JSON null. The scala-sttp (sttp3) generator omits absent optional fields (its hand-rolled encoders build fields via t.field.map(...)), as do json4s and jsoniter-scala defaults - so clients migrating sttp3 -> sttp4 silently start sending explicit nulls, which strict servers reject for non-nullable optional properties. Append .mapJson(_.dropNullValues) to the generated circe encoders (plain models, inline oneOf members, and the discriminator-configured sealed trait encoder), restoring sttp3 parity: None => field omitted. Note: as in scala-sttp, Option cannot express an explicit JSON null for 'nullable: true' properties; representing null-vs-absent distinctly would need a tri-state type and is out of scope. Fixes OpenAPITools#24360 Made-With: pi
|
thanks for the PR |
|
tested locally and the change looks good |
|
if no one has any feedback, i'll merge it later this week |
|
@wing328 @vivekmahajan do you think it would be better to make this behavior configurable? |
|
Thanks @wing328 @nikhilsu . I'd vote to keep it as-is (non-configurable). This just brings sttp4 back in line with what sttp3 already did, so it's really restoring the old default rather than adding new behavior. A flag wouldn't help much anyway for the case you'd actually want it for — sending an explicit null to clear a field — since Option can't tell "absent" from "null" regardless. That'd need a proper tri-state type, which feels out of scope That said, happy to open a follow-up PR if you'd rather have the toggle. lmk |
|
just merged it. thanks for the contribution. @nikhilsu please test with the latest master to see if it breaks any of your existing use cases. thank you. |
Fixes #24360.
The circe branch of the
scala-sttp4generator uses plainderiveEncoder, which serializesOptionNoneas JSONnull:The
scala-sttp(sttp3) generator omits absent optional fields — its hand-rolled encoders build fields viat.tag.map(v => "tag" -> v.asJson)— as do the json4s branch (json4s omitsNoneby default) andscala-sttp4-jsoniter(transientNonedefaults to true). So clients migrating sttp3 → sttp4 with circe silently start sending explicitnulls, which strict servers (e.g. FastAPI/pydantic non-nullable optional fields) reject with 422.Fix: append
.mapJson(_.dropNullValues)to the generated circe encoders — plain models, inline oneOf members, and the discriminator-configured sealed-trait encoder — restoring sttp3 parity:None⇒ field omitted. The wrapper-composition oneOf encoders delegate to member encoders and inherit the fix. Verified end-to-end: with this changePet(name = "rex", tag = None).asJson.noSpaces == {"name":"rex"}and the regeneratedscala-sttp4-circepetstore sample compiles.Known limitation (unchanged from scala-sttp):
Optioncannot express an explicit JSONnullfornullable: trueproperties; distinguishing null-vs-absent would require a tri-state type and is out of scope here.Tests:
Sttp4CodegenTest#verifyOptionalFieldsOmittedWhenNonewith a new fixture (sttp4-optional-fields.yaml). Samples regenerated via./bin/generate-samples.sh bin/configs/scala-sttp4*.yaml— only the circe sample changes (json4s/jsoniter are unaffected, confirmed by zero-diff regeneration).PR checklist
master.Summary by cubic
Omit optional fields set to None in
scala-sttp4circemodels by dropping null values during encoding. This restoressttp3behavior and prevents sending JSON nulls that strict servers reject..mapJson(_.dropNullValues)to all generatedcirceencoders (plain models, inline oneOf members, and discriminator-based sealed traits).verifyOptionalFieldsOmittedWhenNonewithsttp4-optional-fields.yaml.scala-sttp4-circepetstore samples; other libraries (json4s,scala-sttp4-jsoniter) unchanged.Written for commit 2a3769c. Summary will update on new commits.